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The entries in this annotated bibliography are 
divided into four aajor sections. The first includes soae of the 
arguments pro and con on the cloze procedure and a fev research 
studies along vith sone articles that give **hov-to*' recoBnendations. 
The second aajor section includes five subsections: the first deals 
with basic research concerning the validity and reliability of the 
cloze procedure as a device for assessing second language 
proficiency; the second includes a few references to cloze in tLe 
teaching of languages; the third is concerned with cloze as a ■easure 
of readability for ESL/EFL aaterials; the fourth treats cloze as an 
elicitation procedure for interlanguage data; and the fifth discusses 
oral/aural cloze research. The third wajor section includes 
references on cToze tests applied in languages other than English and 
also to social varieties of English. The last aajor section includes 
a subsection on general reviews of cloze research, followed by a 
subsection on reviews of cloze research applied to second language 
proficiency aeasureaent. (Author/PBP) 
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RLSIlARai WriH CLOZE PIIOCLUURU IN MEASIJIUNG THE 
PUOFICIKNCY 0»' NON-NATIVU SPEAKERS OF HNGLISH: 
AN ANNOTATED BIBLIOGftXPIIY 



llavclock Ellis once said that every artist writes his own 
autobiography. Perhaps ho would draw the line at bibliography. 
However, even a bibliography reflects the author's bias In its 
selectivity. If it is annotated, as this one is, the bias is apt 
to be even more apparent. Since the flurry of research with the 
close procedure which began in the late 1960s and which has con- 
tinued even to the time of this writing in 197S» the cloze pro- 
cedure has become a classroom notion, if not a household word. 
Miiny teachers of language, including English as a second or foreign 
language, and bilingual educators have begun to talk about and 
cxperincnt with the close technique. It is probably true, none- 
theless, that the technique has been vastly underexploited to date. 
For this reason, it is hoped that the research cited here will 
provide guidelines for further research and experimentation, as 
well as applications. 

Typically, the procedure has been used for measuring the proficiency 
of students in a classroom situation, or in an institutional testing 
context; it has sometimes been used for the investigation of certain 
leamer or material characteristics. With respect to learner 
characteristics, the technique is eminently applicable to the 
study of so-called errors or interlanguage phenojaena, yet only 
a handful of such studies can be cited in this bibliography. The 
cloze procedure provides a ready basis for focusing on particular 
sorts of grammatical rules and characterizing either leamer 
grammars or the effects of teaching strategies. Concerning mate- 
rial characteristics, the cloze procedure has often been used to 
deal with questions about readability, or the difficulty of 
aurally presented material. 

There are many ways of constructing and administering cloze tesfs, 
and there are methods of combining the cloze procedure «*:ilh other 
testing or data clicitation procedures, e.g,, ^^Iwlation (see the 
references to S. Johansson below). However, most studies have 
utilized the standard procedure of deleting every nth word 
(usually every Sth, 6th, or 7th) from a passage of written prose. 
The alternative of deleting selected parts of speech, or "content" 
words as opposed to "function" words, etc., has also been used. 



hut less frequently. The possible combinations of the two proce- 
dures and the various possibilities within each are nearly endless. 
Nevertheless, all clo^e studies have one feature in corawon: they 
require a subject or group of subjects to replace missing verbal 
material in sunie context. 

The bibliography which follows is divided into four major sections. 
The first includes some of the arsuiaents pro and con and a few 
research studies along with some articles that give "how-to" 
recommendations concerning cloze procedure, llie heavy debate is 
represented principally in the articles by Rand and myself in that 
section. 

The second major section includes five subsections: tho first deals 
with basic research concerning the validity and reliability of close 
procedure as a device for assessing second language proficiency; 
tho second includes a few references to close in the teaching of 
languages; the third is concerned with cloze as a measure of 
readability for liSL/EFL materials; the fourth treats close as an 
olicltation procedure for interlanguage data; and the fifth 
discusses oral/aural cloze research. 

Tlie third major section includes references on cloze tests applied 
in languages other than English and also to social varieties of 
English. 

The last mtijor section includes a subsection on general reviews 
of cloze research, followed by a subsection on reviews of cloze 
research applied to second language proficiency neasurcnent. 



Pros and Cons on t he Basic Procedures-Research and Debate 

Allen, Virginia French. Toward a thumbnail test of English competence. 
Tt-SOI. Quarterly , 1968, 2, 240-242. 

Recommends' a blank- in-soi.tcnce procedure for teaching and 
testing HSI./F.FL. Includes a recommendation concerning cloze 
procedure. 

Anderson, J. Cloze Reading Co")prchension. Port Moresby, Papua- 
New Guinea: Governmant Prii.ter, 1970. 

A test which enables iricrs to equate different groups in 
terms of their rcspectitrv reading abilities. Utilizes a mathemat- 
ical formula for detormiviini* readability coefficients of reading 
selections. 

Bowen, J. Donald. TES. I. riu-.earch for the classroom, TESOL 
Quarterly , 1972, 6, >«;i-6^*. 

Reviews a number of research projects conducted at the 
University of Califomi i nt Los Angeles. Includes a number of 
references to cloze te-*-i r\ search. 
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Clark, John I rsychonict r ic ctnis iJrrat ions anJ lanRuap.c testing, 
hi SpolsKy, H. (IaI,), { un-fnt hcnds in l.at>.i'.u;<ii',f Tcstin?; * in 
prt'parat ten, 

Cloro uriKfJiirt' i- cit *d as an "inJirect mcasuru" of language 
prof iciiMu> . Clark rocDiimcnas that it bs? validated against more 
direct mcasurt's, 

Cuu';in, K.D, C h»2V*f^n^jn M|ue for infonnant participation in 
t'st.il»l i sliinj: rofjister. Uiipuf)! i sht-d manuscript, n.p., n.d. 
Proposes the po.s'.ihi I ity that cJore procedure might be 
sensitive to rt-j-.ister d i f f\ voncei;, 

Cai'v ia-ramtw, M.iric, and Krowitz, Nfary J. Contextual testing and 
coiiiaiinicat ivo com}H'tence. Pape^ prercntod at the Colden Annni- 
vcrsary Mi-tting of thtr Linguistic Society of /Xmcrica, New York, 
nccember Ii)74. 

Criticises standardized tests of language proficiency in- 
cluding Ci;i;r, MAGU, and TOI-FL. Suggests a "contextual i zed" 
task-based test which the authors believe will be superior to cloze 
prucfdure and other techniques. 

Could, Judith, and Spencer, Ja>Tiie. Critical evaluation of the cloze 
test procedure. Unp.iblished manuscript, American University at 
Cairo, 1974. 

• Criticizes the contextually acceptable method of scoring- 
it is not woll defined, according to the authors. The cloze test 
employed contained 38 blanks and was correlated with the various 
part scores on the Michigan Test of l-nglish Language Proficiency, 
Forms F and H, llie study used 105 subjects, but occasionally 
reports results with groups as small as 8. No significant differ- 
ences were fourjd between exact -word and acceptable-word scoring 
methods. (The f.«ct that the results here contradict Stubbs and 
Tucker, 1974, suggests a replication.) 

Cradman, Harry L. Fundamental considerations in the evaluation 
of foreign language proficiency. Paper presented at the 
International Seminar on Language Testing, San Juan, Puerto 
Rico, May 1973. 

Reviews the literature on language testing in relationship 
to findings in linguistics and psycholinguistics— particularly 
in first and second language acquisition studies. Argues for a 
more comprehensive theory of linguistic competence in order to 
explain language test data. Insists on tests that are related to 
the context of communication in ways that arc as natural as possible. 
Mentions findings concerning the cloze procedure and other integra- 
tive tests. 

Groot, Peter J.M. Validation of language tests. Paper presented 
at the International Seminar on Language Testing, San Juan, Puerto 
Rico, hUiy 1975. 

Discusses fill-in-blank procedures anc multiple-choice ver- 
sions of language tests. 




ItiKram, Ktiiahetlu lln* psychcil inguist it basis. In Spolsky^ 

PtvuMits •! luiinher i>t dM untiuiis oi the term psychuliti- 
ijuistics aiul ruIatiN thvm to practical intt rprvtat ions of lanp^ua^^e 
testings especially clu;ie procctlure. Will be of intercht pri- 
murily as a thoroi^^li review of the theoretical literature. 
Hinhlijihts some research finUinus» but mnkes no attempt at complete- 
ncss* 

Lynch» V.IK Clorentrupv: A technique Tor studying audience 

response to films. Jnpuhlished Ph*l>. dissertat ion^ University 
of Iowa, 11»?2* 

Filmj^ were interrupted and 92 subjects were tested on their 
capacity to supply niissini* information. ITie technique seems 
applicable to the measurement of differences in subject ability in 
comprehending filing. (Possible applications to second language 
situations where audio-visual techniques are used are obvious.) 

Mcleod^ and Anderson, J. Readability assessment and word 
redundancy of printed English. Psychological Reports , 1966^ 
IjB, 35-38. 

Posits a mathematical relationship between the estimated 
redundancies of passa{»es of Hn^l ish prose. The estimates arc 
based on independent samples of the same population of subjects. 
Claims that there is a linear relationship between the logarithms 
of estimated redundancies of clqze passa^^es tested against inde- 
pendent samples of children at corresponding grade levels. Con- 
cludes that "the present experiments suggest that meaningful 
estimates of infotnnation rate, behavioraUy derived by cloze test 
procedures may prove as feasible--and potentially as productive— 
a«; have e';tlnated sample variances of obtained test scores** (p. 38). 

— - — . Development of a standardised reading test designed to 
discriminate effectively at the adolescent Ir^vel. Mimeograph, 
University of Saskatchewan, Saskatoon > Canada ^ 1973. 

Reports the development of cloze-type tests suitable for 

children with reading ages ranging up to ten-and-a-half years. 

Kminently applicable to second language testing problems. 

Moss> Kenyon. Cloze testing in reading comprehension. TESL 
Re|>orter , 1072, 6, 3. 

Reports research on the use of the cloze procedure to evaluate 
reading comprehension of 7th and 8th graders. Includes a brief 
bibliography on the cloze procedure and second language testing. 

Oiler, J.W., Jr. On the generation and modification of grammars. 
Paper presented at the Summer Meeting of the Linguistic Society 
of America, Amherst^ Massachusetts, July 1974. FORl tM, in press. 
Cites the high degree of variance overlap between cloze 
tests, essay writing, oral interview, various forms of paraphrase 
recognition, and completion tasks as evidence in favor of an under- 
lying conceptual dependency network of some sort which might be 
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charactcrizi'd us a grammar of t^xpectancy. Arn"«-*s that the in- 
vest i^at ion of I.innu.it.t' tosts. like c\oz>: juoctJure, mi^'.ht provide 
much useful infonnut ion as an c»l ic i tat ion device for thy invesii- 
^•ation of developing j'.raumars at various stages, 

• f'rap.iaatics inU lanRuage testing. In Spolsky» B, (Kd,), 

Current Trends iti I.att{'.uat*e Test int;, i n p re pa ra t i on . 

Provides a brit*t" ii'storical overview of the development of 
pragnatics and develops the notion of a pnt' ic expectancy 
firanunar in terw.s of the intorpret.«t inn of ai .voinplary sentence, 
A primitive example of a fragment of an expectancy grammar provides 
a model in terms of which to explain performance on cloze tests 
and other integrative language tests, 

, Hxpectancy for successive elements: Key ingredient to 

languige use. Language Annals , 1974, 7, 443-452, 

Discusses the notion of an expectancy graiainar as a basis for 
explaining the variance which is conunon to many integrative tests, 
including clo^e procedure, 

. Integrative and discrete-point tests at UCLA: A reply. 

Workpapers in TlISl: UCU , 1973, 7, 85-94, 

A critique ot the article by Rand entitled "Integrative and 
Discrete-Point Tests at UCLA," In particular thi» paper reinterprets 
Rand's factor analysis of close procedure and a number of other 
tests in the Connolly study, 

. Discrete-|>oint tests versus tests of integrative skills. 

In Oiler, J,W,, Jr,, and Richards, Jack C, (lids.). F ocus on 
the Learner: Pragmatic Perspectives for the. Language Teacher, 
Rowley, Mass.: Newbury House, 1973, 184-99. 

Argues that integrative tests, like cloze procedure, dictation, 
translation, and certain typos of multiple-choice items, such as 
sentence piraphrase or selecting the central idea of a paragraph, 
are superior to discrete-point tests which focus on phonemic 
contrasts, particular morphological endings, and the like. 

Pack, Alice. Cloze testing and procedure. TESL Reporter. 1973, 
6, 1-2. 

Outlines techniques for constructing, administering, and 
evaluating cloze scores for liSL purposes. Cites correspondences 
between frustrational , instructional, and independent reading levels. 

Pickett, G.D, A comparison of translation and blank filling as 
testing techniques, English Language Teaching , 1968, 23, 21-6. 
Presents an argument against blank filling (a special variety 
of cloze procedure) and in favor of translation. 

Rand, Karl. Integrative and discrete-point tests at UCLA, Workpapers 

in TiiSL: UCLA , 1972, 6, 67-78. 

Argues that earlier research using cloze procedure and dicta- 
tion failed to establish the superiority of integrative tests. 
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such as cloze aiiJ dictation, over Uiscrt'to-iwiiit tests (actually, 
BHiltiplc-choicc tests of readiiij;, grammar, and vocabulary). Rand 
reports a factor analysis from the Connolly. Masters thesis, which 
used a four-scale aural interview, in addition to four cJoze tests, 
four dictation passages, and two composition iteos {a sentence 
completion task and a paragraph completion task). Kand found that 
one factor accounted for 70 percent of the variance and received 
loadings from all 21 of the variables included. 

. Some evidence for the predictive validity of the ESLPE, 

Workpapers in T(SL; ttCU . 1974, 8, 51-59. 

Reports on a preliminary validation of the ESIPU Winter 
1974 Version (W74). The test includes the following subsections: 
a 15-point multiple-choice gramiaar section focusing on typical 
foreign student difficulties; a 15-point gramaar section concen- 
trating on "non-standard dialect problems"; a 25-blank cloje test 
(items "ranged from five to eight words apart" and were selected 
by the test constructor); a 20-point reading test consisting of 
wultiple-chuice items on three passages; a 2F-i>oint listening test 
involving multiple-choice items based on a short lecture and a 
ten-line dialogue; and two dictations worth 15 points each, 
consisting of about 125 words each. Rand concludes that the test 
is a good placement device correlating highly with teacher ratings 
of students tested. 

Spolsky, B. What does it mean to know a language: How do you get 
someone to perform his competence? Paper presented at the 
Second Conference on Problems of Foreign Language Testing, 
University of Southern California, November 1968, In OUer, 
J.W., Jr., and Richards, Jack C. (Eds.), Focus on the learner; 
Pragmatic Perspectives for the Language Teacher . RowleyT 
^lass.: Newbury House, 1973, ld4-76. 

Proposes close procedure along with the noise test and a 
number of other integrative tests as measures of "overall language 
proficiency." This paper is one of the early arguments in favor 
of what is currently being called "global language proficiency 
testing," 

Taylor, W,L, Cloze procedure: A new tool for measuring readability. 
Journalism Quarterly , 1953, 30, 415-433, 

Tlie first published statement concerning the "cloze pro- 
cedure" developed by Wilson L, Taylor. Originally the procedure 
was conceived as a readability measure for passages of prose. 
In this study, evidence and arguments are presented for the supe- 
riority of a measure like cloze procedure over traditional read- 
ability formulas such as the Flesch or Dale-Chall formulas, 

, Recent developments in the use of cloze procedure. Journalism 

Quarterly, 1956, 33, 42-48, 99. 

Reports on ii.-*carc»» '»moloying cloze procedures for listening 
comprehension tests and also suggest* th^ nossibility of using the 
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ctoae pntceJure for testhij* foreign litnKu.if:«» proficiency. An 
Oct' n cut semi lull article, 

— , and Uh-» K.W, lixiHTititiutal rcajers lend u hand to science: 

♦H'io2f" metliod works in written Korean and my serve ns a tool 

for Korean latjf'.ii.ifje reforin. Korean Mossenger , 1«.>54, 5, 4-5. 
See Taylor (IUSO), alMve. 

Upshur, John A. Productive coitiuiunicat ion testing: A progress 
report. In Pirrin, CI ,, and Trim, .I.L.M. (Eds.)» A npH cations 
of t.tti;',ui St ics; Selected {\ipers o f lite 3nd Internat lon ai 
jrllB' ^.^ - Apl > n e J iTHifni sties. Oanihridgc, l:nr;land, 1 969 . 
AlsiTTn OlKr, "j.'iV, , Jr., and liichards, J.C, C^js*0» Focu s 
on the Learner; Pr a j^j tt^i c Perspectives for the LmguaRC Teacher* 
kowley, Mass, : Newbury lluuse, 



1 1 . Researc h with Cloze Proccdure'-nspecial ly with Non»N*ativo Speakers 
of Hnp.l i sh 

A. As g Testing Pcvicc 

Anderson, J, Application of a cloze procedure to learners of 
Hnglish as a foreign language. Unpublished Ph.D. dissertation, 
University of New England, 1969. 

I'ifty-itctn tests achieved reliabilities of .95 and better. 
Concludes that close technique is a more sensitive measuring 
instrument than the judgments of reading exports. There were also 
several independent checks of validity which indicated cloze scores 
are valid measures of rcadin)', comprehension ability for non-native 
speakers of Ilnglish. This is a thorough, milestone research 
project in the evaluation of close procedure and its applications 
in the teaching of P.nglish as a foreign language. 

— — , Research on comprehension in reading. In Bracken, D.K., 
and Melmquist, D. (lids.), Improving Reading Ability Around 
the World . Third International Heading Association World 
Congress on Reading. Newark, Delaware: International Reading 
Association, 1970, 

Another report on blank length. Also investigates the 
feasibility of having the students write all answers in a column 
on an answer sheet in the margin. Revealed that both versions of 
close procedure correlated well with the Katts Sentence Test, 
Actually the test using exact-length blanks correlated slightly 
better with the V/atts Sentence Test than did the test using 
standard- length blanks. Subjects were non-native speakers of 
bnglish in the fourth, fifth, and sixth grades in Papua-New 
Guinea schools. 
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I»it*st'Utt'J .It iIk- ThirU lfjtt'rn;it n.h.i I Kt-.ulinj; Assvuintioii 
Horia Cofijtri ;:; on UiM»Jiiijt, Syaiic-v , Austral iu, Aujiust, 1970, 
lliU 111 Ul.% 

Ui'{H»rtt'J on tlie use ui blanks oi* the word di'Jt'tt'il, Close 
tesis const riiftcil by this procedure worked as well us close tests 
constructed by tin* stands ril prrcedure ol' usin^ a bJatik of uniform 
lennth throU)»luiui , A practical implication is that teuchurs will 
be able to construct cloze tests by usinj; photocopy ing iacilltles. 
Used nou-native sptukers of* Inglish, 

. A technimie for moasnrlnj; readhij: comprehension and readability, 

t.nttli'^h l.an^'u.tj;t» Teach im;. 1971, 2^, 178-182. 

A descriptive statement of the procedure and a very brief 
attempt to descrif»c a model. Hie model harks back to Taylor 
(IUS3), who claimed the technique was a fiood device for testing the 
Roodness of fit lu'tween a speaker or writer's system and a reader 
or listener's system, Reports that blanks of the same length as 
the deleted word work as well as blanks of a standard length. 
Clains there is no difference between ranking of passages or subjects 
for the two blank procedures. 

. Application of a clo2o procedure to English learned as a 

foreign lanj'.uap.e in t?.c Territory of Papua and New Guinea, 
tinglish {.annuat^e Teachi ng. 1972, 27. 6l>-72, 

lixplores severil different. scorTng procedures; exact word, 
Words that make sense in the context, and words that arc of the same 
grammatical class as the deleted word. The result is that the 
relative ranks of the students and tests remain about the same, 
regardless of the procedure used. This replicates findings with 
native speakers. See also Oiler, 1972, on the same topic. 

Howen, J. Donald, A tentative measure of the relative control 
of l.nglish and Atiiharic by 11th grade Ethiopian students. 
Workp apers in TI.SL; UCAA , 1909, 69-89. 

A pioneering study in the application of cloze procedure 
to the ujeasurement of proficiency in English as a second language 
and also bilingual dominance. Although this study did not incorporate 
validity checks in terras of additional tests, comparisons between 
native and non-native performance on the English close tests 
revealed expected differences, and scores in English were generally 
correlated with course grades (rank orders ^vere checked). Scores 
in /Viiharic seemed less systematic (perhaps due to different scoring 
criteria) , 

Browning, Cari. Testing pronunciation indirectly: An experiment. 
Unpublished M,A. thesis. University of California, Los \nficles, 
1974. 

Correlates the UCIA ESLPE (which includes a section of cloze) 
with judgments of nativeness. 
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Cohoit, Aii.li'i'w I), All u|uljtej rupurt oi\ the linijliuh us a Second 
UiiiKu i^e I'i.ictMtit'iii !..\.iiiiiii.it ion. <v'orK{>;tju'fs in 'tt.St.; HCt.A , 

Hrti'l ly mont it'll'", why the i:\ozti techniques were retaiiieU in 
Cohen's fulluw-uji research and further develupment of the UCLA USLPE, 
IMscuHses tlie ratiunale behind inclusion of vuriuus parts* on the 
UCLA, daliiiii from the fall uf 11»T2. 

and Hruwn, Virnini:!. Readinij in u second or foreign 
luni'.uaite. In Spi»lsK>, B, (t.d.). Current Trends in Lant'uagc 
Tes t i n^t; , in prep.ir.it ion. 

l»escri!>es a v.iriety of reading'- item typf.s, including cloze 
procedure, lor the measurement of proficiency in a second or foreign 
lantiuaRo. Trovidcs a fairly ihoroujjh review of the literature. 

Connolly, ILiroId V. The oral HnRlisih proficiency uf foreign 
students at UCLA, Unpublished M.A. thesis. University of 
California, Los Anjicles, 1972. 

Cross-validates an oral Interview procedure against cloze 

tests and dictation, 

Cunrad, Christine. The cloze procedure as a measure of English 
proficiency. Unpublished M.A. thesis, University of California, 
Los AnRcles, 1970. U'orkpa|>crs in TCSL; UCLA , 1971, 5, 159 
(Abstract), 

Surveys literature on the use of clozo procedure, especially 
in the tcachinj: or testing of Hnglish as a foreign language. Reports 
data from a correlation of an experimental cloze test with the UCLA 
Lnglish as a Second Lanj'.uage Placement Lxaminution, Form 2C 
(stiminarized and expanded statistically by Oiler and Conrad, 1971). 
Tlie perfornance of the cloze procedure as a discriminator between 
different ability levels was al.so checked against the performance 
of students at various levels in the English as a Second Language 
Program at UCL\ and at a Junior college in the Los ^\ngoles area, 

Crawford, Alan, The cloze procedure as a measure of reading com- 
prehension of elementary Icve» Mexican-American and Anglo- 
iXnierican children. Md.f), dissertation, UCLA, 1970, 

Cloze procedure differentiated among Mexican-Americans- dom- 
inant in l:nglish, dominant in Spanish, and Anglo groups, Crawford 
compared the cloze tests results against other tests including the 
Stanford Achievement Test and Comprehensive Teits of Basic Skills 
Level 2. Form Q, Some subjects at both third- and sixth-grade 
levels also took the dilmore Oral Reading Test, Form C, Tl»e cloze 
tests proved to be valid and reliable with both groups of subjects, 

narnell, Donald K, The development of an English language profi- 
ciency test of foreign students using the clozentropy procedure. 
Speech Monogrnph s, 1970, 57, 56-46, 

Compares performance on a cloze test against the Test of 
Fnglish as a Foreign Language for 48 foreign students at the 
University of Colorado, The cloze test was also administered to 
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200 native speakers of linjtlish at the sarae University. The close 
test was about as reliable as the TOl M. C.Hb). Hie correlation 
between the two tests (total scores) was ,83, /\n analysis of 
variance revealed that the content and difficulty of test nuiterial^ 
major subject area of examinees, and level and tiiajor of the native 
cowparison group had significant influences on the index of Rnglish 
proficiency. Scoring was based on the degree of conformity of non- 
native responses to the norius established on the basis of the 200 
natk-'e speakers. This meant it had to be done by computer, which 
makes the technique somewhat cumber sortie for classroom purposes. 
The hivht;.t correlation between the cloze test and any part of the 
TOHFL was with listening comprehension, 

Davies, A, Two tests of speeded reading. Paper presented at tho 
language Testing Symposium, Georgetown University, Washington, 
D.C., March 1^74, 

Failed to replicate certain earlier findings with cloze pro- 
cedure. However, Uavies used a test construction method that relied 
on experimenter judgments rather than standard every nth word 
deletion, 

Estrada, F,X, The effect of increasing syntactic complexity on 
reading comprehension. Unpublished M.A. thesis. University of 
Southern California, 1969. 

Investigated the performance of 75 fourth-grade Navajos and 
native speakers of English on cloze tests of differing levels of 
difficulty. Results indicated that cloze test difficulty is signif- 
icantly related to the complexity of sentence structures included. 
The native ilnglish speakers out-performed the Navajos on all levels. 

Evans, Gordon, and ILiastrup, Kirsten. Experiments with cloze 
procedure. Department of English, Danmarks Laererhojskole, 
Endrupborg, 1973. Submitted for publication. 

Discusses the development of proficiency tests in English 
by a Nordic test-development group set up in 1971. The article 
"deals with two of the subtests, both of which use a modified 
clczo procedure" (p. 3). The test finally included was the result 
of pre-testing in which items with low discrimination indexes 
were eliminated. The final draft of the cloze portion included 
17 items with an average item discrimination of .53. The cloze 
test portion requires 9 out of the 51 minutes devoted to the written 
section of the test. Also, a partial dictation of the sort used 
by Johansson is included in the listening comprehension section. 

Goldmann, Tamar T. English as second language testing and the 
prediction of academic success. Unpublished ^f.A. thesis. 
University of California, Los Angeles, 1972. Workpapers in 
TESL: UCL\ , 1972, 6, 97 (Abstract). 

Compared" correlations between part scores and the total score 

of the English as a Second Language Placement Examination at UCLA 

against grade point average. The correlations were between .08 

and ,23- -too slight to be of much predictive value. 
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Hufnian, John !•. Asscssraent of Inglish proficiency in the African 
primary school, lliipuhl i slu'd research report, Haifa University, 
Israel, 1974. 

Reports tho development of close tests for elementary school 
children in grades j-7 in African primary schools. The tf • wore 
too difficult for the second-j'.radc level, but gave satisfactory 
results at higher levels. Item-to-total correlations ranged between 
,8S and ,9S. Individual item reliabilities ranged from .60 to 
.91, Hofnian concludes thai tlie procedure is generally applicable 
at the elementary grade levels, but the particular tests which he 
devised worked best with students in the fourth through seventh 
grades. 

Hunt, AJ!. Factor validity of clo:c tests of reading comprehension 
of P.nglish learned as a foreign language. Unpublished manuscript. 
University of New Hngland, 19(»9, 

Hndeavored to establish the factor validity of use of cloze 
tests with students for whom 1-nglish is a foreign language. Two 
hundred and eighty-four elementary school pupils of standards four, 
five, and six completed clo^e tests with every eighth word deleted. 
Multiple-choice comprehension questions were made over the same 
material to test vocabulary, knowledge of stated facts, grasp of 
organisation, perceiving causality, identifying the main theme, 
and making inferences. One factor accounted for 63 perce»it of the 
variance in the matrix, a factor called 'reading comprehension.* 
This study supported those of Bormuth (1966) and Anderson (1969). 
Tlio close procedure measures skills closely related to or 
identical to those measured by a multiple choice comprehension question 
test (Riley, 1973, p. 21). 

Inal, Novin, Testing knowledge of prepositions by using the close 
procedure. Unpublished M.A. thesis. University of California, 
Los Angeles, 1972. Workpapers in TESl: UCLA , 1972, 6, 
99-100 (Abstract). 

Four questions were asked: (1) Is a cloze test of prepo- 
sitions feasible, reliable, and valid? (2) Are response distri- 
butions for native and non-native speakers similar or different? 
(3) Will contrastive analysis enable the researcher to predict 
responses from a grrup of students from the same native language 
background? (4) Will the tost provide useful diagnostic information 
concerning trouble spots for non-native speakers in general? 
All questions wore answered in the affirmative except for (3). 
Only a small percentage of errors were predictable on the basis of 
contrastive analysis. 

Irvine, Patricia; Atai, Parvin; and Oiler, J.W, , Jr. Cloze, dictation, 
and the Test of linglish as a Foreign Language. Language 
Learning, 1974, 24, 245-52. 

The Test of Hnglish as a Foreign Language was administered 
to 159 non-native speakers of I-nglish in Iran. They also took a 
close test and two dictations. Part scores and total scores were 
intcrcorrelated between the TORFL and the two experimental measures. 
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Results confirm earlier findinjis. The clozv^ and dictation correlated 
better with the listen inji comprv^hcnsion part of TOlil L than any other 
subsection of that examination. Ilie authors conclude that the data 
provide strong support for integrative tcstinp. procedures. 

Karitt^ki, Albert. Student achievement la language and subject matter 
in vwo Kt-nyan primary school curricula. Unpublished M.A. thesis. 
University of California, Los Angeles, 1972. Worki)aitcrs in 
Tl-SL: UCL\, 1972, 6, 100 (Abstract). 

The experimenter administered the Certificate of Primary 
Education Examination in mathematics, English, and general knowledge 
plus cloze tests in linp.lish. Subjects were students in Kenyan 
schools wlio did not speal; linglish as a native language. Correla- 
tions between the clore tests and the CPU Unglish examinati'^n 
revealed "concurrent validity." 

Kirn, Harriet. Tlic effect of practice on performance on dictations 
and close tests. Unpublished M.A. thesis. University of California, 
Los Angeles, 1972, IVorkpapors in TESL: UCU , 1972, 6, 102 
(Abstract). " 

1S5 foreign students in USL classes at UCLA in the fall quarter 
of 1971 and A4 nSL students in university extension served as subjects 
for this study. Kighty-five of the students in the regular ESL 
classes practiced taking either a cloze tost or dictations throughout 
the quarter. Pre-test , post-test results indicated that there 
was no difference in the improvement rate of the experimental groups 
as compared to that of the control groups. A follow-up experiment 
with students at the lower level in extension classes indicated 
that there may be some improveipcnt when cloze passages and dictations 
are focused on particular "phonetic and grammatical points," 

Nagato, Naoko. The effect of I'LHS on EFL achievement in secondary 
school. Unpublished M.A. thesis, University of California, Los 
Angeles, 1972, Workpapers in TESL: UCIJV , 1972, 6, 105 (Abstract). 
Compares the performance of Japanese students at the secondary 
level who have experienced English as a foreign language in the 
elementary school against thos? who have not. Three cloze tests 
were administered. A response frequency analysis for each item 
on all three tests is included along with other statistics. 

Oiler, J.W., Jr. Assessing competence in ESL: Reading. TESOL 
Quarterly , 1972, 6, 313-23. 

Advances the hypothesis that the high correlations among 
tests of listening comprehension, speaking, reading and writing 
are an indication of test validity. Discusses correlations between 
cloze tests and dictation. 

, Scoring methods and difficulty levels for cloze tests of 

proficiency in English as a second language. Modern Unguage 
Journai , 1972, 56, 151-158. 

Investigates five different scoring methods and concludes 

a slight superiority of an acceptable-word scoring method over the 
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exact -word scoring method. The acceptable-word method counts as 
correct any response which docs not violate any of the con- 
straints of the passa^je. Item analyses, intorcorrclation matrices, 
as well as first, second, and third order partial correlations 
are provided for clo20 tests compared with the UCLA English as 
a Second Language Placement Examination, Form 2A Revised. 
Reliabilities for the three cloze tests included in the study 
ranged from .95 to .99 on the Kudcr-Uichardson formula 20 for 
subjects, and from .80 to ,V5 on the same formula for items. 
The study utilized 398 subjects. There was a tendency for clo2© 
tests to correlate better with the sections of the UCLA IiSLPi: 
which were tnore integrative in nature, 

, Dictation as a test of l-SL proficiency. In U.K. Allen 

and R.N, Campbell (Uds,), Teach in.q Cnnlish as a Second Language: 
A Book of [Readings, New YorTi McGraw-Hill, 1972, 346-354. 

Compares correlations between dictation tests and cloze tests 

and the UCLA English as a Second Language Placement Examination, 

Form 2A Revised, and dictation correlated with UCLA ESLPE, Form 

1 only, 

, and Conrad, Christine A. The cloze technique and ESL 

proficiency. Language Learning , 1971, 21^, 183-195. 

A cloze test scored by the exact-word procedure is corre- 
lated with the UCLA ESLPE, Form 2C. The multiple correlation 
between all sections of the ESLPE and the cloze test was .88, 
Surprisingly, the cloze test correlated best with the dictation 
subsection on the UCLA ESLPE, This replicated a similar finding 
of Darnell, 1968, with the TOEFL. 

, and Inal, Nev^n. A cloze test of English prepositions, 

TESOL Quarterly , 1971, 5, 315-26, 

Every other prepositTon was deleted from a passage of prose 
in English, The test was taken by 19 native speakers of English, 
53 native speakers of Turkish, and 110 foreign students entering 
UCLA from a wide variety of language backgrounds. Item analyses 
and response frequency data are reported. Certain intralingual 
confusions were exhibited, regardless of language background. 

, and Nagato, Naoko, The long-term effect of FLES: An 

experiment, ^todern Language Journal , 1974, 58, 13-18. 

Reports results from an experiment using cloze procedure to 
measure the proficiency of Japanese secondary school students, 
some of whom had had training in English as a foreign language in 
the elementary grades, and some of whom had not. Non-FLES students 
apparently are able to overtake FLES students by the eleventh grade. 

Pike, Lewis. An evaluation of present and alternative item formats 
for use in the Test of English as a Foreign Language. Mimeo- 
graph, Educational Testing Service, Princeton, New Jersey, October 
1973. 

This 114-page report constitutes the most thorough validity 
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study on the Test of HnBlish as a Foreign LaiiRuaso ever undertaken. 
It investigated correlations betuven sections and sub-sections of 
the lOIiFL, as well as intercorrelations with various experimental 
tests, including a paragraph rewriting task developed by Kellog 
Hunt, a standard close test (open-ended), several multiple-choice 
sub-tests (including a multiple-choice version of a cloze test), 
an essay, and an oral interview. Tlic results, on the whole, ptovide 
additional support for integrative over discrete-point tests, and 
replicate a number of findings using the cloze procedure. 

Rufener, J.B. Use of the cloze procedure with Thai school children: 
An exploratory study of readability and individual differences 
in reading. Unpublished Ph.D. dissertation, University of 
Illinois at Urbana-Champaign, 1972. 

Urban and rural Thai children, grades two to five, 151 boys 
and 142 girls, were the subjects in this study. Raven's Progressive 
Matrices, class grades in reading, and average academic grades were 
used as assessment measures. Passages from texts in use were se- 
lected from grade levels one to six and mutilated by an every- 
fifth-word deletion. Subjects completed cloze tests at their own 
grade level and the next level above and below. Exact-word scoring 
was used. Results showed that cloze passages were ranked consist- 
ently among subjects, indicating that cloze assesses readability. 
Cloze scores increased with grade level and ability, indicating 
that cloze is sensitive to differences in reading ability. Cloze 
scores correlated positively with school achievement and scores 
on Raven* s Progressive Ntatriccs, indicating the validity of cloze 
tests for assessing Thai elementary school children (Riley, 1973, 
p. 25). 

Spolsky, Bernard, Reduced redundancy as a language test tool. 
Paper presented at the Second International Congress of Applied 
Linguistics, Cambridge, England, September 1969. ERIC 
ED 031 702. 

Develops an argument for glohrJ testing procedures. Mentions 
the cloze technique as modified in the Spolsky et al, noise test. 

Stevenson, D.K. Construct validity and the Test of English as a 
Foreign Language. Unpublished Ph.D. dissertation. University 
of New Mexico, 1974. 

Investigated oral cloze procedure and part scores on TOEFL, 

Finds no evidence of unique variance that can be associated with the 

Listening Comprehension sub-test which is not also associated with 

Reading Comprehension and other sub-tests. 

Stubbs, J, Bartow, and Tucker, G, Richard. The cloze test as a 
measure of ESL proficiency for Arab students. Modem Language 
Journal , 1974. 58, 239-41, 

Correlates the American University at Beirut English Entrance 
Examination with a cloze test scored by exact-word and acceptable- 
word methods. Concludes that the cloze test in general is a 
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highly valid method of assessing overall lanftuage proficiency. 
Replicates the findinns oi' Oiler, 1972, and OllJr and Conrad. 
1971. 



B. As a Teaching Procedure 

Eskey, David. A model program for teaching advanced reading to 
students of English as a foreign language. Language Learning. 
1973. 23, I()i>-184, f 

Discusses some of the psychological processes underlying 
reading skill. Recommends the use of cloze procedure as a testing 
device to measure readability and comprehension and also as a 
teaching procedure which will hopefully force "the hesitant student 
to think, to attack his reading problems actively by drawing on 
his several kinds of knowledge about English" (p. 182). 

Freidman, Mildred M, Hie use of cloze procedure for improving 
the reading comprehension of foreign students at the University 
of Florida. Unpublished Ph.D. dissertation. University of 
Florida, 1964. 

Claims validity for cloze both as a aeasure of comprehension 
and as a teaching device— though cloze was not more effective than 
other techniques— and compares cloze with Metropolitan Achievement 
Test Heading Section and with multiple-choice tests of reading 
comprehension. 

Plaister, Tod. Teaching reading comprehension to the advanced 
ESL student using the cloze procedure. RELC Journal, 1973. 
4, 31-38. "~ 

Discusses the reading process and argues why the cloze 
procedure should be a suitable device for enriching the learning 
experience of students in ESL/EFL classes. 

Schaeffer, H.H. A vocabulary program using "language redundancy." 
Pittsburgh University, January 1961, ERIC ED 014 879. 

Used cloze procedure in helping to teach German function 
words. The German words were inserted in a passage of English 
prose at appropriate points. After the training program, 
students were able to translate correctly 60 percent of the 
German words. 



C. As a Measure of Readability 

Anderson, J. Selecting a suitable "reader": Procedures for 
teachers to assess language difficulty. REIC Journal. 1971. 
2, 55-42. — 

Describes the use of cloze procedure "as a measure of reading 
difficulty with second language learners" and "how teachers may 
construct their own cloze tests, administer and score these, and 
interpret the cloze scores for grading reading materials" (p. 36). 
A very readable and helpful "how-to" article. 
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«— — . Ifio d'«v("U»[M)vnt of a reading laboratory for second language 

Discusses i\n- ik'velopmcnt of a reading iaboratory modeled 
after the Sci^rntUic Ueseurch Associates reading lahoratoricii. Used 
the clo2c pro-edare to grade 210 selections in terms of reading 
difficulty. 

, and Hu'it, A. if, A frarae of reference for cloze tests of 

readability of Hniilish learned as a foreign language. Paper 
presented to tiio 42iid ANZAAS Congress, Port Moresby, Papua- 
New Guinea, 1«70. 

Invest I tiated che correspondence of cloze test scores with 
"traditional ivvels of reading— independent level, instructional 
level, and frurtrational level." Reveals that 53, 44 to 53, and less 
than 44 percent correspond respectively to the three preceding 
levels. Thii. vj,s determined with cloze tests scored in a standard 
way— count inij only exact-word responses as correct. 

Haskell, John. Using cloze to select reading material for the ESL 
classrooni. Paper presented at the Eighth Annual TESOL Convention, 
Denver, Colorado, March 1974. 

Reported satisfactory results of grading materials with 

cio?,e procedure. Concludes that the exact-word scoring method is 

preferable (for reasons of economy of time and effort) for purposes 

of estimating readability. 

. Refining the cloze procedure for English as a second 

language. English Record, 1974. 



. Selecting reading materials: A new approach. Idiom. 

1972, 3, 1-4. . 

Humphreys, J.W., and Key, R. The use of cloze technique to grade 
tests in the Pacific Horizons Reading scheme. Papua New Guinea 

Journal of Education , 1971, _7, 5-8. 

Close tests were administered to two classes of 35 pupils each 

at the elementary and junior high levels in Papua-New Guinea. 

Results indicated sensitivity to such subtle variables as style 

of writing, degree of conformity to children's beliefs, and 

interest value of the materials. 



D. As an Elicitation Device 



Bebout, Linda Joyce. An error analysis comparing the ability of 
learners of English as a second language to extract information 
from written material. Unpublished Ph.D. dissertation, Cornell 
University, 1974. 

Used a cloze- like procedure for the elicitation of data from 
9- to ll-year-old Anglo children and from native Spanish-speaking 
adults. Uie study concentrates on questions about the usefulness 
of the cloze procedure as an elicitation device and also presents 
a thorough analysis of the rating of responses according to their 
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ucci'plahility by a group of adult native speakers of linglish. 
iTovidcs insight into the bridge hetweon inter 'anguagc analysis 
and proficiency testing. 

Sec a lso the Johansson papers under section U.K.; Oiler, Bowen, 
Uicn, and Mason under section lU; and Oiler and Inal under section 

^* Aur^UOral Cloze Test Research (with native and non-native 
speakers) 

ClarK, John I, I). Theoretical and technical considerations in aural 
proficiency testing. Paper presented at the Language Testing 
Symposium, Georgetown University, Washington, D.C., March 
1974. 

Discusses "thcorotical and practical considerations in th© 
development and use of aural proficiency tests" (p. 1), Reviews 
research with "direct" and "indirect" techniques of language 
proficiency measurement, Clark considers the cloze procedure an 
indirect measure. 

Craker, H.U, Clozentropy procedure as an instrument for measuring 
•oral Hnglish competencies of first grade children. Unpublished 
Ph.D. dissertation. University of New Mexico, 1971. 

Revealed that an aural close procedure used with children 
was capable of discriminating among sociolinguistic backgrounds 
for Anglo, Spanish, Black, and Navajo subjects. 

Dickens, M., and Williams, F. An experimental application of 

close procedure and attitude measures to listening comprehension. 
Speech Monographs, 1964, 3J[, 103-8. 

Used two aural cloze tests where messages recorded on tape 
deleted a word every five seconds by means of random masking noise. 
The cloze tests were reliable at .70 and .80 and intercorrelated 
at ,73. Attitude toward the message (one "persuasive," the other 
"expository") proved to be insignificantly related to either cloze 
tests or to the multiple-choice tests of comprehension used, 

Gradman, Harry L,, and Spolsky, Bernard, Reduced redundancy testing 
A progress report. Paper presented at the Language Testing 
Symposium, Georgetown University, Washington, D.C, March 1974. 
Replicates earlier findings with a noise test reported by 
Spolsky and challenged by Johansson. Also replicates findings of 
Oiler and Johansson with straight dictation and partial dictation 
(that is, dictation combined with cloze procedure). Recommends 
the noise test, oral cloze tests, and straight dictation as measures 
of second language proficiency. 

Gregory-Panopoulos, J.F. An experimental application of cloze 
procedure as a diagnostic test of listening comprehension among 
foreign students. Unpublished Ph,D, dissertation. University 
of Southern California, 1966, 
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Compares the porfurwanct of im aur.il close procedure listing 
cvcry-f ifth-wurd deletions from a t:ipe-rccorUcd tiies.sa}«e (white 
noise was substituted for blanks) ufiainst the Brown-Carl sen Com- 
prehension Test» lorn aM» and California Reading Tcj*t» Advanced, 
Form Y. Concludes that cloac procedure is a valid and reliable 
measure of listening comprehension. The aural cloze reliability 
was ,92 and .93 for subjects at USC and Cambridge Adult School, 
respectively, 

Johansson, Stig, An evaluation of the noise test: A method for 
testing overall second language proficiency by perception under 
nuisKing noise. International Rev iew of Applied Linguistics, 

1973, n, 107-13T. 

A sTTghtly expanded and revised version of the paper presented 

in Copenhagen, 1972 (see below). 

. Controlled distortion as a language testing tool, Pap^ 

presented at the Third International Congress of Applietfi^inguistlcs, 
Copenhagen, Denmark, August 1972, 

Criticizes the Spolsky noise test. Claims that parti^ dicta- 
tion is superior to the noise technique. 

- — -« Partial dictation as test of foreign language proficicincy. 
Report Number 3, Swedish English Contrastive Studies. Lund 
University, 1973. • 

Utilises a testing procedure that combines dictation with cloze 
procedure. The student reads a passage with blanks while he hears 
the complete passage dictated. The task is to fill in the blanks. 
This procedure was recommended by Brooks (196S), although JohZsson 
developed it independently on the basis of a study of dictati^ 
at UCIj\ and some of the research on cloze procedure listed in this 
bibliography* Johansson provides a useful and interesting anal)%is 
of errors and their types. ^ 

Kaplan, Robert B., and Jones, R.A, A cloze procedure test of listening 
for university level students of ESL, Mimeocrapl^ University 
of Southern California, 1970. 

Explores the possibility of using an oral version of cloze 

procedure at the university level. Conclusions were not very 

encouraging. 

Nutter, Barry Lynn. Presentation methods, deletion patterns, 

and passage types for use with aural cloze. Ed.D. dissertation. 
University of Arizona, 1974. Dissertation Abstracts, 1974, 

35, 945.A. 

Among the presentation methods were techniques using a 
preview and repetition* 311 students were tested at a junior high 
In Tucson, Arizona. Passage type, presentation, method, and 
deletion all proved to be significant factors by an analysis of 
variance. 
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Spolsky. B.; SiRurd, Henst; Sato, M. ; Walker. I-.; and Artcrburn, C. 
I»ri»limii!ary studies ii, the development of techniques for testing 
overall second lan^uajto proficiency. In J.A. Upshur, ajid J, 
Fata (lids.), pro!) 1 ems in Fore inn Language Testing , lanRuaRe 
tearninj'. , 1968, Special Issue No. 3, 79-1017 

Reports experiments with the noise technique. Includes 

inerestini; cumincnts by participants in the conference— especially 

Upshur on cloze procedure. 



^ ^ ^ • R<'?^ parch with UnguaRcs Other Than Hnglish or Social Varieties of 
iiusl ish 

Anderson, J., and Rahman, F. A feasibility study of applicability 
of close procedure as a measuring device for reading compre- 
hension* in the Malay language, Singapore: Ministry of Education, 
NRC Number 7. 1972. 

Five passages in htalay were ranked by reading specialists 
and teachers. Every fifth word was deleted, with five versions 
over each passage so that each word in every passage was deleted 
in at least one of the versions. Pupils at grades four, five, and 
six were tested. The tests had high re-test reliabilities and 
discriminated well among subjects. 

Bondaruk, John; Child, James; and Tetrault, E, Experiments with 
contextual testing. Paper presented at the Language Testing 
Sywposiujn, Georgetown University, Washington, D.C., March 1974, 
Samples of contextual test forms. Reports on the application 
of a modified cloze procedure to tests in English, French, Spanish, 
Chinese, Russian, and Portuguese. The research was sponsored by 
the Department of Defense and focused on the testing of aptitude, 
achievement in language courses, and overall language proficiency. 

See also Bowen, 1969, on Amharic. 

Carroll, J.B.; Carton, A.S.; and Wilds, CP. An investigation of 
cloze items in the measurement of achievement in foreign languages, 
Cambridge, Mass,: College Entrance Examination Board, 1959, 
The study was designed to investigate the feasibility of 
using cloze-type items for College Board foreign language tests. 
Reviews the literature and discusses the development of cloze tests 
in English, French, and German, Test characteristics, including 
reliabilities and validity measures, are reported. An experiment 
on the possibility of aural cloze is also included. It is note- 
worthy that the test preparation did not follow standard cloze 
procedure of deleting every nth word from a passage of prose. 
Rather, certain items were selected on the basis of pre-testing. 
Concludes that the cloze technique measures language skill quite 
indiref;tly and that there is very likely a specific skill in taking 
cloze tests which is not correlated very highly with language 
proficiency. The study has two major limitations: first, it 
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utitizca a very small sampU' for many of its Reneralizallohs; 
and secuud, innttsul of usiriK a proficiency ineasiurc as a 
validatiiii'. basis, the study usi»d the MLAT, The exact-word scorinfi 
method was used. 



Cooper, Unbcrt L. , and t-ishman, Joshua A. Soim? issues in the 
theory and measurement of lanRuage attitude. Paper presented 
at the International Seminar on Language Testing, San Juan, 
Puerto Hi CO, my 1973. 

Used a cloze test in linglish and in Hebrew which deleted 
every sixth word from passages in the respective languages. The 
intent of the rnithors was to use language proficiency as a criterion 
against which various attitude measures could be compared. No 
results are reported in this paper, however. 

Dicn, Ton That. A measure of language competence using the cloze 
procedure. Workp apers in THSt.; UCLA , 1972, 6, 94-9S (Abstract). 
Compares peruTniiance on cloze passages basFd on original 
language prose with passages based on translation. Contrasts 
native language perforiiuince with non-native performance. Seeks 
infomuition about specific learner errors and their causes. Uses 
Vietnamese and Hnglish as test languages. 

Kerr, A.H, Psycho linguistics and a new empirical approach to the 
weasurencnt of comprehension and readability of French as a 
second language. In Siiapkins and Miller (Hds.), Chan gin g 
F.ducation . Sydney, Australia: McGraw-Hill, 1972, 278-293, 
The study concludes that cloze tests in French as (a second 

language have the same applicability as in English, 

Klare, George R. ; SijKikiao, R.W.; and Stolurow, L.M, The cloze 
procedure: A convenient readability test for reading materials 
and translations, Arlington, Virginia: Institute for Defense 
Analyses, 1971. 

Reports good success with cloze procedure in Vietnainese. 
Cloze scores are apparently sensitive to the quality of trans- 
lations, 

Lcong, S,N, Cloze procedure as a measuring device for reading compre- 
hension in the Chinese language. Singapore: Ministry of Education, 
NRC, No, 4, 1972. 

This study parallels the Anderson and Rahman study discussed 
above. Five passages were selected, and five different versions 
of cloze tests were constnictcd over each passage by deleting every 
fifth word until every word was systematically deleted in each 
passage. Cloze rankings agreed well with rankings by experienced 
teachers. Tlie tests discriminated between teachers and correlated 
with language examinations. Counting acceptable-word responses and 
exact-word responses in scoring yielded an intercorrelation of 
.98, It was concluded that cloze is as "suitable a measure of 
reading comprehension in Chinese as in European languages** (Riley, 
1973, p, 25), 
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McLfud, J. t.'ujuparat ivi' assessment of rcadinu couiprchcnsion: 
A five-count ry stuJy. Saskatoon, CanaJui Institute of Child 
Guidance anil hfvelopmt iit , Mimeoi«raph» li»74. 

Compares performance of children in five languages (Czech, 
Polish, French, Spanish, am! linnlish) and concludes that the clozo 
procedure is in fact "translatable." Comparable results can bo 
achieved acro.s:; languages. See olso Oiler, Bowcn, Dicn, and Mason. 
1972, 

Kfason. Victor. l.anijua}'.e competence as measured by the clozo tost. 
Unpublished M.A. thesis. University of California, Los Angeles, 
W« nk])apcrs in IT.Sl.t UCf.A , 1972, 6, 103 (Abstract). 
Keports detailed analysis of cloze tciTts in linfilish and 
Thai. Compares cloze test scores with grades from students* 
standardized final Unglish exams, and with scores of u group of 
$$ American high school seniors who took the BngUsh versions of 
the tests, lliis study was coordinated with the thesis by Oicn 
completed in the same year. Reports detailed response frequency 
analysis for items on the BngUsh close tests, 

. Report on cloze tests administered to Thai students. 

Bulletin of the Hnglish hanguat»e Center , 1972, 2, 51-50, 

Summary of Mason's M.A. thesis, listed abovcT This report 
also includes full copies of the tests in English and Thai. 

Oiler, J.W., Jr.; Bowen, J. ttonald; Dicn, Ton That; and Mason, 

Victor W. Cloze tests in English, Thai, and Vietnamese: Native 
and non-native performance. Language Learning , 1972, 22, 
1-1$. — 
Among other things this study investigated types of errors 
made by native and npn-native speakers of English. Mean scores 
for native and non-native speakers on translated passages indicated 
rough comparability between the original passage and its trans- 
lation. Native speakers made fewer responses which did not conform 
to at least some of the contextual constraints in the passages. 

Potter, T.C. Reading Comprfchcnsion among Minority Groups; Child 
Generated Instructional Materials. 1968, ERIC ED 031 546, 
Stories generated by third-grade children who spoke Black 
English were more readable, as judged by the cloze measure, than 
the Standard English stories commonly used in the classroom, 

Cohen, Andrew D, English language placement testing: Separating 
foreign English from minority English. Paper presented at the 
Eighth Annual TESOL Convention, Denver, Colorado, March 1974, 
Mentions cloze procedure along with several other testing 

techniques which might be useful in fulfilling the goal stated in 

the title. 

Schaier, Barbara; Cooper, Robert L.; and Fishman, Joshua A. 

Language, technology, and persuasion: A Middle Eastern example. 
Research Committee on Sociolinguistics, Language in Science 
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liJucation, Hebri'w lltsiversily of Jtrusaltm, 1974. 

Ri'puits tiikUiuis froiii the CtHjpcr and !it.hman urticlc li-.itcd 
ahuve. tU'os not nientiun any result ^; from cluzt- procviUiiv. Vw 
Buijur rosttlt was that Hebrew turneJ out to ho more effective for 
arguiiii: in scieiuific terms, while Arabic wu.s wore effective for 
persuasion in "traditional tnesjiajjes." 

Shiba, SuKeyori. A study of readability incasures--uppHcations of 
the cloze procedure to the Japanese lanj'.u-'Ke. Jn i>uiese Journal 
of Psyeholoj'.y. li)57, 27, 07-7.^ (linglish Abr.tract). 

Concludes that close procedure is a viable nieasur;' of read- 
ability in Japanese, 

Sinaiko, II. W., and Klare, 0,R. Further experiments in Jatiguage 
translation: A second evaluation of the readability of computer 
translations. Arlington, VivKiniu: Institute for Defense 
Analyses, Science and Technology Division, I'apcr P-80S, 1971. 
Close tests of Vietnamese translated from iinglish proved 
valid and reliable tor evaluating the adequacy of translations. 
Three types of translation were compared: computer, edited computer, 
and huraan. The study suggests that "cloze may prove a useful measure 
of growth of skill in reading a foreign language" (Riley, 1973, 
p. 25). 

Stcndahl, Christina. The relative proficiency in their native 
language and in linglish showi\ by Swedish students of English 
at university level. English Institute, Gothenburg University, 
1972. 

Used the close procedure to explore the relationship between 
native and foreign language proficiency in Swedish and English 
respectively. Found, a surprisingly high correlation between students* 
proficiency in the two languages. That is, students who tended to 
score well on tests in Swedish also tended to do well on the tests 
in Lnglish, and vice-versa. 

Wijnstra, Johan M., and Van Wagcningen, Nico, The cloze procedure 
as a neasure of first and second language proficiency. Submitted 
for publication ( Dutch Journal of Psychology ), 1973. 

Used cloze procedure tests in Dutch and Spanish. A aiultiplc- 
choice version of one of the tests in Dutch was constructed. The 
result was that the multiple-choice form correlated at ,85 witfh the 
open-ended standard cloze procedure scored by the acceptable-word 
method as recommended by Oiler (1972). Replicates a number of findings 
from previous studies. 

Williams, F,, and Wood, B, Negro children ^s speech: Some social 
class differences in word predictability. Language and Speech. 
1970, 13, 141-lSO. 

UscT"clozc procedure to demonstrate that receptive competence 
for children of lower socioeconomic status is not necessarily 
bidialectal. 



22 



ERIC <U 



Reviews 



A. Uem^ra 1 

Bennett » Stanley, et al. Hie cloie prucedure"a review of the 
literature tmd ponsible icut ion < to the study of deviant 
language functiuris. In Studies in I.in^^unge a nd t.an>;uafic 
beha v i or : P ro «» res!^ R ep ort l\\ Ana Arbors UnivcrsTty of 
Fficlunan Center for Kesearch on Lauiiuage and LanguaRc ilchavior. 
miC Ul) 017 y^O. 

Bickley. A.C.; nUington, BilUc J.; and Bickley, Rachel T, The 
cU»2e procedure: A conspectus. Journal of Ueadinp Behavior . 
1970, 2. 232-2 J9. 

Borrouth, J.R, Cloze readability procedure. Occasional Report 
No. 1. Center for the Study of F.valualion of Instructional 
Programs, University of California, Los Angeles, 1967. 

Boyce, M»4x K. A coniprchensivo bibliography of the cloze pro- 

dure. State College of Victoria at Toorak, Victoria, Australia, 
1974. 

Greene, P.P. Close symposium. 17th Yearbook of the National 
Reading Conference. Milwaukee? National Reading Conference, 

1968, no. iSU 

Suggest that cloze procedure might be used to measure sccorid 
language proficiency. 

Hafner, Lawrence E. Relationships of various measures to the 
"cloze." In Rric L, Thurston and Lawrence E. Hafner (Eds.), 
The Philosophical and Sociological Bases of Reading , 13th 
VcarhiHiK of the National Reading Conference. Milwaukee: Nr.tional 
Read ing Conference, 1965, 135-145. 

Jongsma, E. Tho cloze procedure as a teaching technique. Newark, 
Delaware: International Reading Association, 1971. 

McLeod, J,, and Anderson, J, An approach to assessment of reading 
ability through information transmission. Journal of Reading 

Behavior , 1970, 2, 116-143. 

Provides an excellent review of research relating cloze scores 
to information theory and derived estimates of redundancy. Defines 
an uncertainty reduction index in terras of cloze tost scores which, 
among other things, can be used for estimating an individual's 
readin^i ability. Suggests the possibility of applying uncertainty 
reduction index estimates in the investigation of the translatability 
of cloze tests across languages. Implies that the same estimates 
of uncertainty reduction could be applied to the English as a foreign 
language context. 
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Potter, T.C. A taxonomy of cIozk' resoardu part 1: Readability 
and road inj; coii!|nvhcns ion, Injilcwood, Calilornia: Southwestern 
Rei;ioi)al Laboratury for Lducational kescarch and Development, !968, 

Runktn, {;,F., Jr. Tlie cio^e procedure: A survey of research. 
Hth Yearbook of the Nation.! I Reading C onference . Mi 1 waukcc : 
National Keadinn Conterence, li'65, 153-lSO. 

Robinson^ I'.U. An i ntroduction to the clo:e procedure; An 

annotat ed hibTu)Jraphy. Newark, Delaware: International Reading 
Association, iy72. 

Weaver, W.W. Tlieoretical aspects of the close procedure. 14th 
Yearboo k cif the National Reading Conference. Mi 1 waukee I 
National Reading Conference, 1965, 115-132. 

Weintraub, S. The cloze procedure. Reading Teacher , 1968, 
21^, 567-571, 607. 

B. Language Testing and Cloze Procedure 
See especially the annotated bibliography by Riley, below, 

Gradiiian, ILL. A bibliography of research on language testing. 
In Spolsky, B. (Ed.), Current Trends in Language Testing, in 
preparation. 

This is probably the most .comprehensive bibliography on the 
topic in existence. 

. Reduced redundancy testing: A reconsideration, i'aper 

presented at the ATBSOL Conference, Dublin, Ireland, Sniamer 
1973. In O'Brien, Maureen Concannon (Ed.), Testing iv. second 
Language Teaching: N'ew Dimensions . Dublin, IrelandV^AlESOL, 

Reviews results with the Spolsky et al. test since 1969. 
Construes it as an aural cloze test. Concludes that more research 
is needed, hut the method remains substantially usefui. Gradnan 
suggests its use as an elicitation device for the investigation 
of error t>T>cs common to "a particular level of language pro- 
ficiency" (p. 47). 

Groot, Peter J.M. Ntodern language testing in Western Europe. 
In Spolsky, B. (Ed.), Current Trends in Languap.e Testing, in 
preparation. 

This report mentions several studies in Europe that are 
using some version of cloze procedure. Several of the reports 
mentioned arc not published, 

Jones, R,L., and Spolsky, B, (Eds.), Testing Language Proficiency: 
Theoretical and Practical Is.^ucs , Arlington, Virginia: Center 
for Applied Linguistics, in press. 

This volume includes papers presented at the Language Testing 

Symposium held in Washington, D.C, March 1974, at Georgetown 
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University, Items of interest are referred to separately in the 
present bibliography, 

Kennedy^ Graeme 1). The testing of comprehension. In Spolsky, B. 
0'*i»)> Current Trends in Lani;uat|e Testin?^ , in preparation. 

This 127-page manuscript is a monumental contribution to the 
understanding of listening comprehension and reading comprehension. 
An excellent and very thoroughgoing review of the research is 
included. Naturally, the subject is broad and the review cannot 
include all references, but it is remarkably complete. Kennedy 
also takes account of research with close procedures and dictation 
along with a great many other testing procedures which he capably 
describes. 



Klare, George R. Assessing readability. Readi ng Research Quarterly, 

1974-75, jO, 62-102. ^ 

Excellent review of the research on close as applied to 
readability. Includes recent work with non-native speakers of 
English and with languages other than English, 

Oiler, J.W., Jr. Cloze tests of second language proficiency and 
what they measure. Paper presented at the section on 
Examinations, Tests, and Certificates, Third International 
Congress of Applied Linguistics, Copenhagen, Denmark, 1972, 
Language Learning , 1973, 23, 105-118, 

Reviews recent literature on cloze tests, especially in 
second language proficiency measurement, and suggests some unanswered 
research questions. Argues that cloze procedure taps an underlying 
competence which can be explained in terms of a pragmatic grammar 
of expectancy, 

. Language testing. In H, Douglas Brown and Ronald Kardhaugh 

(Eds,), Survey of Applied Linguistics . Ann Arbor: University 
of Michigan, in press. 

An introductory survey of language testing research and findings. 
Traces the roots of discrete-point testing to contrastive analysis 
theory and structural linguistics. Develops a definition for the 
notion "efficiency of the learner's internalized expectancy 
grammar." Cites research with cloze procedure and other integrative 
techniques as well as error analysis studies which support the 
proposed definition. 
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, Pragmatic language testing. Language Sciences, 1973, 

28 , 7-12, 

Intro?»uces some notions from a pragmatic theory of language 
and extrapolates their relevance to practices in language testing. 
Argues that cloze procedure is very closely related to other 
integrative language tests, such as oral interview, dictation, 
and reading tests of various sorts. 
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. IVstinn lanjiuaf.e skilU, In K.R. Shijier and Betty NlaUsen 

(Hds.), I. anKuanc in Ame rican Iruliati Education . Salt Lake City; 
University of Utah', Ii>7J. 

A non-technical introduction to a numbex' of testing procedures, 
including discrete-point tests on iirammatical structure, sentence 
intcrpretat ion, word order, vocabulary, subject-verb agreement, 
structural parallelism, case of pronouns, comparison of adjectives, 
formation of adverbs, and formation of irregular verbs. Also 
discusses integrative tests of reading skills, including vocabulary 
in context, sentence paraphrasing, identifying the central idea of 
paragraph, cloie procedure dictation, and composition, 

» and S|>olsky. B, The Test of Unglish as a Foreign Language, 

In B, Spolsky (lid,). Current Trends in Language Testing, in 
preparation, " 

Surveys briefly the history of the development of the Test 

of linglish as a Foreign Language and research isolating to it. 

Correlations between the TOEFL and close tests are reported, 

Riley, Pamela M. The cloze procedure— a selected annotated 

bibliography, Lae: Papua-New Guinea University of Technology, 

Includes sections on review articles, general methodology 
and rationale, readability, teaching English as a second language, 
languages other than Unglish, and other miscellaneous studies, 
Rilcy»s bibliography is consider.ably broader in scope and less 
focused than Robinson's bibl iogr^f hy (cited in section IV.A. 
above) , 

Spolsky, B, Linguists and language testers. In B, Spolsky (Ed,)» 
Current Trends in. La n guage Testing , in preparation. 

Discusses the history of language testing and argues in 
favor of integrative tests like cloze procedure. Stresses the need 
for tests that are sensitive to sociolinguistic variables. 
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^* Preparing an d Usinn Self-Teaching Units for Foreign LingtiaRcs, 
by {leraid h. Logan, 1974. Available from ERIC Document 
Reproduction Service (EDRS) : ED 098 808, 

^* A Selected Bihl ioRraphy on Bilingual/Bicultural Education , 

by James W. Ney and Donella K. Eberle, li)75. AvoTIab 1 e 
from EDRS: ED 098 813. 

5. A Selected Bibliography on Language Teaching and Learning , 
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Available from EDRS: ED 100 189. 
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by Paul T. Griffith, 1975. Available from EDRS: 
ED 100 183, 

5, Working Papers in Linguistics . 

by Tim Shopen, 1975. Available soon from EDRS. 
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by Stephen Cahir, Brad Jeffries, and Rosa ^fontes, 1975. 
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by Stephen L. Levy, 1975. Available soon from EDRS. 

8* A Selected Bibliography of Films and Videotapes on Foreign 
Language Teacher Training , ~" 

by Peter A. Eddy, 1975. Available soon from EDRS. 

9. ERIC Documents on Foreign Language Teaching and Linguistics: 
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by Peter A. Eddy, 1975. Available soon from EDRS. 

10» Effects of Social Situation on Language Use; Theory and 
Application , 

by William Check, Theodore B. Kalivoda, and Genelle Morain, 
1975. Available soon from EDRS. 

U. Radio in Foreign Language Education , 

by Robert J. Nelson and Richard E. Wood, 1975. Available 
soon from EDRS. 

12. Error Analysis in the Classroom . 

by Patricia B. Powell, 1975. Available soon from EDRS, 
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CA».*HRIC/CLL SHRinS ON LANmVGES AND LINGUISTICS 



I'.lUC (Hducational Resources Information Center) is a nationwide 
network of information centers, each responsible for a given 
educational level or field of study. ERIC is supported by the 
National Institute of Education of the U.S. Department of 
Health, Hducation, and Welfare. The basic objective of ERIC 
is to make current developments in educational research, instruc- 
tion, and personnel preparation more readily accessible to 
educators and members of related professions. 

The HRIC Clearinghouse on Languages and Linguistics (ERIC/CLL), 
one of the specialized clearinghouses in the ERIC system, is 
operated by the Center for Applied Linguistics. ERIC/CLL is 
specifically responsible for the collection and dissemination of 
information in the general area of research and application in 
languages, linguistics, and language teaching and learning. 

In addition to processing information, ERIC/CLL is also involved 
in information synthesis and analysis. The Clearinghouse com- 
missions recognised authorities in languages and linguistics to 
write analyses of the current issues in their areas of specialty. 
The resultant documents, intended for use by educators and 
researchers, are published under the title CAL* ERIC/CLL Series on 
Languages and Linguistics . The series includes practical guides 
for classroom teachers, extensive state-of-the-art papers, and 
selected bibliographies. 

The material in this publication was prepared pursuant to a 
contract with the National Institute of Education, U.S. Depart- 
ment of Health, Education, and Welfare. Contractors undertaking 
such projects under Government sponsorship are encouraged to 
express freely their judgment in professional and technical 
matters. Prior to publication, the manuscript was submitted to 
the Teachers of English to Speakers of Other Languages for 
critical review and determination of professional competence. 
This publication has met such standards. Points of view or 
opinions, however, do not necessarily represent the official 
view or opinions of either the Teachers of English to Speakers 
of Other Languages or the National Institute of Education. 

This publication will be announced in the ERIC monthly abstract 
journal Resources in Education ( RIE) and will be available from 
the ERIC Document Reproduction Service, Computer Microfilm Inter- 
national Corporation, P.O. Box 190, Arlington, Virginia 22210. 
See RIE for ordering information and ED number. 

For further information on the ERIC system, ERIC/CLL, and the 
CAL* ERIC/CLL information scries, write to ERIC Clearinghouse on 
Languages and Linguistics, Center for Applied Linguistics, 
1611 North Kent Street, Arlington, Virginia 22209. 



